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Abstract: The legislative and poUcy 

background for evidence-based practice 
(EBP) is presented with implications for 
research on assistive technology (AT) research 
in general and for the AT consideration 
process specifically. Requirements for the 
development of research-based evidence of 
AT effectiveness are presented as a guide for 
researchers and developers of AT and for 
practitioners who use findings of AT 
effectiveness. EBP as part of decision-making 
guidelines for AT consideration are presented. 
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In today’s educational environments, demands 
for evidence-based practice in assistive 
technology (AT) decision-making are being 
articulated with increasing frequency by 
administrators, policy-makers, researchers, 
and classroom practitioners (Dugan, 
Milbourne, Campbell, & Wilcox, 2004; 
Edyburn, 2003; HiU, 2006; Parette, Peterson- 
Karlan, & Wojcik, 2005). Owing in large part 
to legislative influences (i.e.. The No Child 
Left Behind Act of 2001 [NCLB] and the 
Individuals with Disabilities Education 
Improvement Act of 2004 [IDEIA 2004]), the 
term ‘evidence-based practice’ has become 
almost ubiquitous in education circles 
(Detrich, Keyworth, & States, 2007). 


NCLB requires that that educational 
interventions used to improve educational 
performance are based on scientifically-based 
research (Odom et ak, 2005; U.S. Department 
of Education, 2003). Education professionals 
familiar with the legislation are aware that 
there are more than 100 references to 
‘research’ noted in its text, communicating a 
clear intent to have school reform efforts 
focused on educational curricula, instmctional 
strategies, and achievement that are based in 
research [§1114(B)(ii)]. Similarly, IDEIA 2004 
requires that educational interventions with 
students having disabilities are scientifically- 
based instructional practices [118 Stat. 
2650(B)]. 

For a decade now, it has been a mandate that 
AT be ‘considered’ in the development of the 
lEP of every student with a disability [20 
U.S.C. 1401 § 614(B)(v)], and the use of AT 
has been implicitly linked to enhanced 
educational outcomes for students 
[§616 (a) (2) (A)]. Moreover, in school settings, 
AT is an intervention in that it is applied not 
only to enhance or improve student access to 
educational opportunities but also to improve 
performance in the general education and life 
skills curricula (Peterson-Karlan & Parette, 
2007). Thus, educational law and policy 
generated from these laws require that AT 
consideration be based upon evidence of AT 
effectiveness and that claims of effectiveness 
arise from scientifically-based research. This, 
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in turn, requires that we understand what 
evidence-based practice is, what constitutes an 
acceptable scientificaUy-based research claim, 
and how such evidence and research can 
inform the process of AT consideration and 
decision-making. 

What is Evidence-Based Practice? 

While an entire issue of Exceptional Children 
in 2005 was devoted to evidence-based 
approaches in special education, there is still 
no consensus regarding a definition of and 
guidelines for such practices (Detrich et ah, 
2007; Odom et ah, 2005). As noted by Odom 
et ah, numerous groups have developed 
standards for evidence-based practice, though 
there is no agreement across groups regarding 
the quantity or quality of evidence required. 
Recent compilations of evidence-based 
recommendations are observable in the health 
care industry (HealthLinks, 2007; U.S. 


Department of Health and Human Services, 
n.d.), medicine (Wikipedia, 2007a), nursing 
(Beyea & Slattery, 2006; Malloch & Porter- 
O’Grady, 2006), counseling (ChwaHsz, 2003), 
psychology (American Psychological 
Association, 2005), and early childhood 
special education (Smith et ak, 2003; Strain & 
Dunlap, n.d.). These recommendations reflect 
both commonalities and differences in 
thinking about evidence-based practices. 

Despite this seeming lack of clarity in what 
EBP might be conceptually, the U.S. 
Department of Education (2003) is clear in 
how evidence-based practice works. NCLB 
calls upon educational practitioners to use 
“scientifically-based research to guide their 
decisions about which interventions to 
implement” (U.S. Department of Education). 
Interventions are broadly conceptualized to 
include such things as reading and math 
curricula, school-wide reform programs, after- 


Table 1 

Characteristics of Scientific Research from the Education Sciences Reform Act (P.L. 107-279) 

Scientific research studies: 

• Employ systematic, empirical methods that draw on observation or experiment; 

• Involve data analyses that are adequate to support the general findings; 

• Rely on measurements or observational methods that provide reliable data; 

• Make claims of causal relationships only in random assignment experiments [emphasis added] or 
other designs (to the extent such designs substantially eliminate plausible competing explanations or 
competing results); 

• Ensure that studies and methods are presented in sufficient detail and clarity to allow for replication, or at a 
minimum, to offer the opportunity to build systematically on the findings of the research; 

• Use research designs and methods appropriate to the research question posed; and 

• Obtain acceptance by a peer-reviewed journal or approval by a panel of experts through a comparably 
rigorous, objective, and scientific review. 
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school programs, and technologies and are 
criticized for having claims to effectiveness 
that, while being supported by evidence, is 
based upon “poorly designed or advocacy- 
driven studies (U.S. Department of 
Education, 2003). While NCLB allows 
flexibility in the type of “reliable evidence of 
effectiveness” presented [115 Stat. 1597, 
§1 502(B)], there is an emphasis on “rigorous” 
scientific evidence (U.S. Department of 
Education). 

What Is Scientificallj-Fased Evidence? 

Given its explicit connection to EBP, 
knowledge of the characteristics of 
scientifically-based research and 

methodologies used to produce it are needed 
by AT developers, researchers who investigate 
AT effectiveness, and practitioners who must 
now use EBP. The Education Sciences 
Reform Act of 2002 established standards for 
scientifically-based research that include (a) 
applying rigorous, systematic, and objective 
methodologies to obtain reliable and valid 
knowledge to education activities and 
programs; and (b) presenting findings and 
making claims that are appropriate to and 
supported by the methods that have been 
employed. Table 1 details the seven 
characteristics of research that would be 
considered to be rigorous, systematic, 
objective, and reliable. Noted in Table 1 (with 
emphasis added) is what has been referred to 
as the ‘gold standard’ for research design: the 
‘random assignment experiment,’ also referred 
to as the ‘randomized controlled trial’ (RCT). 
The randomized controlled trial has been 
cited as being the highest standard for 
research in medicine, welfare, employment, 
and psychology (Odom et ak, 2005; U.S. 
Department of Education, 2003). While 
“other designs” which “substantially eliminate 
plausible competing explanations for the 
obtained results” are permissible, the RCT has 
been emphatically promoted as one of two 
key indicators of “strong evidence” of 


effectiveness with the second being trials 
showing effectiveness in two or more typical 
school settings, including a “setting similar to 
that of your schools/classrooms”(U.S. 
Department of Education) Randomized 
controlled trials are studies “that randomly 
assign individuals to an intervention group or 
to a control group, in order to measure the 
effects of the intervention” (U.S. Department 
of Education) This would mean, for example, 
that to determine the effectiveness of a text- 
to-speech (TTS) digital reader on text passage 
comprehension, a developer or researcher 
would need to (a) identify a large number of 
students with reading impairments at a 
particular grade level, and (b) randomly assign 
some in the same class to an intervention 
using the TTS technology, while others are 
might be assigned to an intervention in which 
someone reads the text to the student. This 
would be repeated across all the classes at that 
grade level having impaired readers. A lesser 
claim of “possible evidence of effectiveness” 
is also permitted when closely matched 
comparison groups are used in Ueu of 
randomized assignment. A closely matched 
comparison group is created, for example, in 
the TTS technology study when students who 
have reading deficits from one or two 
classrooms, perhaps at one school, are 
assigned to the TTS intervention while others 
having the same degree of reading deficit 
from other classrooms, perhaps in another 
school, are assigned to the adult reader 
intervention. 

Developing Claims for AT Outcomes and 
Benefits 

It is obvious that these standards for 
scientifically-based research and claims of 
effectiveness have great impact upon what AT 
may be considered to be effective, how the 
evidence is a claim of AT effectiveness, and 
what would be included in AT decision- 
making consideration. There has not, 
however, been total acceptance among special 
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education researchers of one ‘gold standard’ 
research methodology (Odom et ah, 2005). 
RCT addresses only one of three possible 
questions that can be addressed by research: 
(a) What is happening? (description); (b) Is 
there a systematic effect? (cause); and (c) Why 
or how is it happening? (process or 
mechanism; Odom, et al.). RCT is a method 
for determining effectiveness. The Council for 
Exceptional Children (CEC) Division for 
Research (as cited in Odom et al.) identified 
four different types of methodologies are 
needed to address these questions when 
developing and evaluating the effectiveness of 
intervention practices: (a) experimental group 
(of which RCT is a part); (b) correlational; (c) 
single- subject; and (d) qualitative. Subsequent 
work has established rationale, characteristics 
and standards for group and quasi- 
experimental (Gersten et al., 2005); single- 
subject (Horner et al., 2005); correlational 
(Thompson, Diamond, McWiUiam, Snyder, & 
Snyder, 2005); and qualitative (Brandinger, 
Jimenez, KUngner, Pugach, & Richardson, 
2005) research methods. Space does not 
permit a full examination of the purpose 
served by each approach or the quality 
indicators associated with each; for further 
information the reader is referred to the 
individual articles cited here. 

Multiple methodologies are needed, not just 
because there are multiple questions to be 
answered but also because research in special 
education, including AT development and 
research, is complex (Odom et al., 2005). 
Additionally, different types of research are 
needed as a field, such as AT, emerges and 
develops (Odom et al.). 

Complexily of AT Research 

Special education research has been 
characterized as the “hardest to do science 
given the local conditions that often limit 
generalization and theory building” (Berliner, 
2002, p. 18). As with special education 


research in general, AT research is inherendy 
complex including the (a) variability of 
participants (i.e., both types and severity of 
disabilities) in service settings [20 U.S.C. 1401; 
§602(3)]; and (b) educational contexts where 
interventions are to be provided [i.e., for 
whom an intervention is designed and in what 
context; Odom et al.; 118 Stat. 2657(29)]. The 
educational context issue is particularly 
problematic given that randomization and 
stratification (critical elements for RCT; 
Gersten et al., 2005; MosteUer & Boruch, 
2002; U.S. Department of Education, 2003) 
may be difficult, if not impossible, due to 
heterogeneity of the student makeup in many 
special education classrooms. Also 
problematic are issues related to low 
prevalence rates for certain groups of students 
(e.g., physical, sensory, severe cognitive or 
multiple disabilities), and clustering of 
students in groups (i.e., classrooms may 
become the units for assignments vs. students; 
Detrich, 2006; Odom et al.). 

Research examining AT development and 
effectiveness shares the complexities cited 
above. Methodology other than RCT may 
better address AT research needs. Carefully 
constructed single- subject designs employing 
individuals as their own controls and 
systematically replicating AT interventions 
across individuals and/or settings within the 
same experiment may be more feasible in 
producing systematic, objective, and reliable 
data regarding outcomes and benefits than 
RCT methodology. Designs including 
multiple baselines across individuals, activities, 
and settings; alternating treatment designs; 
and multiple probe designs among others 
have had a long history in special education 
research (Horner et al., 2005). Concurrent 
time series probe designs can produce 
objective and reliable data concerning long- 
term effectiveness of AT in supporting 
student educational progress (Smith, 2000). 
Such single- subject research designs have 
been instruments of rigorous, scientific 
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methodology for over 40 years (Horner et al., 
2005) and conform to the characteristics of 
scientific research outlined in Table 1, with 
the single exception of RCT design. 

Evidence-Based Practice and the Emergence of AT 
Research 

AT development and research have developed 
over time; technologies, applications of 
technologies, and their implementation in 
service settings emerge over time in a logical 
progression from basic to small-scale applied 
research to large-scale demonstrations. And as 
noted above, not all research appropriately 
addresses the same questions. Descriptions of 
what is happening often precede 
demonstrations of (a) whether specific 
elements cause effects, and (b) how or why 
these effects occur, and (c) different methods 
for obtaining this evidence. Emerging fields 
such as AT use in the school settings may not 
be able to transition from case study to RCT 
research without intervening steps. Using 
science to improve educational or AT 
outcomes may actually be a continuum of 
research activities (Odom et al., 2005) which 
begins with preliminary ideas, hypotheses, 
observations, or descriptions, and then moves 
through classroom-based demonstration and 
design research, and finally culminates in RCT 
studies. None of these activities is sufficient in 
the absence of others; all may be necessary for 
a research-based knowledge base to develop 
that informs educational practice. In the early 
stage of description and exploration of 
specific AT technologies, qualitative research, 
for example, can be used to describe what is 
happening when individuals with disabilities, 
their families, or their educational 
professionals (a) select and use AT; (b) 
examine attitudes, opinions, and beliefs about 
AT consideration, selection and use of AT; or 
(c) examine personal reactions to types of AT 
and AT-supported interventions (Thompson 
et al., 2005). Single-subject research, as 
described above, especially systematic 


replications of AT-supported interventions, 
can provide evidence of AT effectiveness or 
efficiency in school and community 
applications while replications across 
disabilities which differ in important ways 
(e.g., autism spectmm disorders, cognitive 
impairments, learning disabilities) provide 
evidence of the generalizabiUty of AT 
outcomes. 

In summary, it is argued here that, while 
scientifically-based research is certainly 
requisite to the development of evidence- 
based practice, there are methods other than 
RCT that can provide reliable, valid 
descriptions of AT (or AT-supported) 
interventions, examinations of effectiveness, 
and consideration of how they are effective. 
For AT developers and researchers, reliable, 
replicable qualitative, single-subject, or quasi- 
experimental research with carefully 
determined dependent variables and 
consistency of intervention should be used to 
generate evidence-based practice that is 
published in peer-reviewed journals such as 
ATOB. Case studies and building- or district- 
based evaluations of AT effectiveness lacking 
these characteristics, while initially serving a 
helpful purpose, cannot be used as standard 
for evidence-based practice in a maturing 
field. For educational professionals and 
families, awareness of the need for evidence- 
based practice, knowledge of the 
characteristics of appropriate evidence-based 
practice, and application of scientifically-based 
research to AT consideration, selection, and 
implementation are important goals for 
professional development and family 
education. 

Evidence-Based Practice and AT 
Decision-Making Practices: Outcomes 

and Benefits 

Though special education practitioners 
express interest in evidence-based practices 
(CEC, 2007), such interventions are used 
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relatively infrequentiy in classroom settings 
(KratocMl, Albers, & Shernoff, 2004; Odom 
et al., 2005) with little guidance being 
provided to assist families and professionals in 
choosing among available interventions 
(Detrich, 2006). Admittedly, the complexity of 
such guidance is laden with a plethora of 
embedded issues discussed above regarding 
the variability of participants and educational 
contexts 

While the debate about evidence-based 
practice continues, there are still glaring issues 
about the implementation of such practices by 
education professionals. Detrich et al. (2007) 
suggested that there are four pressing EBP 
implementation issues. These include (a) 
effective and accessible dissemination of 
interventions, (b) selection of interventions, 
(c) initial implementation, and (d) 
sustainability. Each of these issues is discussed 
briefly in the following section. 

Effective and Accessible Dissemination of 

Interventions 

Given that many researchers in the field of 
AT are often aligned with university settings, 
publication in peer-reviewed journals is 
viewed as a valued and primary venue for the 
dissemination of knowledge, particularly 
evidence-based findings. However, these 
venues may not be an effective dissemination 
strategy for decision makers (Detrich et al., 
2007). Unfortunately, the very nature of the 
research process is so intensive that findings 
from a body of work are frequently 
distributed across multiple journals. 
Sometimes these journals are in related 
discipline databases such that searches in one 
source (e.g.. Psych Info or Academic Search 


Premier) do not necessarily identify citations 
of archived peer-reviewed publications 
archived in other databases (e.g.. Social 
Sciences Abstracts, ERIC EBSCO). Even 
more perplexing is that the lack of 
accessibility of peer-reviewed publications is 
constrained by subscriptions to online 
journals (i.e., one cannot access the article 
without subscription or payment for the 
article). This is complicated even further by 
the lack of training in conducting searches of 
varying databases where evidence-based 
publications may be archived, coupled with 
lack of training in how to evaluate primary 
source data reported in the articles (Detrich et 
al.). Finally, time constraints on the part of 
decision-makers in intervention settings may 
be such that reading professional journals is a 
low priority activity. 

To meet the needs of the practitioner for 
accessible evidence-based practice, a number 
of Web-based resources have emerged. Table 
2 provides a listing of sites and their URLs 
which provide professionals, consumers, and 
students with organized information regarding 
evidence-based practices. Users of such sites 
must still be aware of and knowledgeable 
about the difference between summaries of 
areas of research and summaries of the 
research itself and the differences in research 
which has been peer-reviewed and that which 
has not. For example, LD OnLine 
(http:/ /www.ldonkne.org/I contains both 
summaries of articles which have undergone 
expert examination (peer review) and those 
that have not. Peer review assures that there 
has been evaluation of the reliability of the 
evidence, the integrity of the treatments or 
interventions, and the validity of the research 
claims. 
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Table 2 

Web-Based Resources Disseminating Information on Research-Based Practices 

“site URL 

What Works Clearinghouse http:/ /ies.ed.gov/ncee/wwc 

Center for Implementing Technology in http:/ /www.cited.org 

Education (CITEd) 


National Center for Technology Innovation http:/ /www.nationaltechcenter.org 


(NCTI) 

Tech Matrix http:/ /www.techmatreix.org 

Center for Evidence-Based Practice: Young http:/ / challengingbehavior.fmhi.usf.edu/ resources.html 

Children with Challenging Behavior 

National Early Childhood Technical Assistance http:/ /www.nectac.org/ topics/ evbased/ evbased.asp 
Center 


Selection of Interventions 

Researchers have noted that systems 

influences impact assistive technology 
decision making (Parette, 1991; Parette, 
Brotherson, & Huer, 2000; Parette, Huer, & 
Brotherson, 2001). Such influences as cost, 
expert opinion, individual preferences (based 
on experiences with particular devices), and 
the effort associated with systems change 
exert powerful influences on decisions 
regarding selections of specific interventions. 
While these influences are indeed practical 
reality, they are valid criteria only to the extent 
that they are applied to a range of possible AT 
solutions for which evidence of effectiveness 
has been established through scientifically- 
based research. One might correctly question 
a decision to provide a student with a 
preferred, less costly, or readily available AT 
tool (for which little valid research 
information is available) in Ueu of one that is 
less preferred, more costly, or which must be 


obtained (but for which evidence of AT 
benefit has been established by an even small 
body of research studies). Thus, professionals 
and families must become consumers of 
evidence of effectiveness when selecting AT 
during a consideration process and either seek 
this information themselves or require 
vendors to provide such information upon 
request. 

Initial Implementation 

In making decisions regarding a particular 
intervention, the question must be asked, 
“What is necessary to gain practitioner 
support?” (Detrich et al., 2007). This may 
present a dilemma in the decision-making 
process since one’s previous training and 
experiences may suggest a particular choice of 
intervention, though pragmatics of 
implementing the decision with practitioners 
may result in a very different choice. 
Decision-makers are often confronted with 
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the challenging issue of how to effectively 
train practitioners. It has been recognized that 
a broad base of competendy trained AT 
practitioners is needed in the field (Parette et 
al., 2005; Parette, Peterson-Karlan, Smith, 
Gray, & SUver-PacuiUa, 2006). However, 
developing such a broad base of effectively 
trained practitioners is more problematic as 
there is Mttie guidance for the field. An 
especially thorny concern is ensuring the 
fidelity of implementation of evidence-based 
findings (Odom et ah, 2005). If fidelity of 
implementation is absent (i.e., the specific 
procedures reported in an evidence-based 
report of an AT-supported intervention are 
not followed), unknown effects may be 
anticipated from what becomes, in essence, an 
unknown intervention (Detrich et al.). 

All too often, decision makers must examine 
available evidence-based practice reports and 
decide if the reported intervention can be 
adapted to meet local circumstances. Detrich 
et al. (2007) described two inherent dangers 
when such decisions are to be made. First, if 
the intervention is changed too much, what is 
implemented is a different intervention for 
which there are no data. Second, if the 
intervention is not modified to accommodate 
local circumstances, it may not be 
implemented at aU. 

Sustainability 

Sustainability is defined as “a characteristic of 
a process or state that can be maintained at a 
certain level indefinitely” (Wikipedia, 2007b). 
Detrich et al. (2007) suggest that sustainable 
programs (a) maintain over time, (b) maintain 
across generations of practitioners, and (c) are 
supported with existing resources of system. 
Wikipedia also notes that sustainability 
“focuses on providing the best outcomes 
[emphasis added] for both the human and 
natural environments now, and into the 
indefinite future.” Unfortunately, evidence- 
based interventions that are not sustainable 


mn the risk of being replaced with alternative, 
ineffective practices. A corollary, then, is that 
the larger the scale of implementation 
required of a particular system, the more 
complex and potentially unsustainable these 
issues become (Detrich et al.). 

Summary 

As CEC (2007b) has noted, evidence-based 
practice, while wanted and needed, is hard to 
find. In the absence of randomized controlled 
trials with large numbers of students with 
disabilities of AT-supported interventions, the 
consideration, selection, and implementation 
of AT in school and community settings will 
need to depend upon AT developers and 
researchers providing systematic, objective, 
and reliable data regarding outcomes and 
benefits based upon research methods 
appropriate to the participant, context, and 
evaluation questions. AT developers and 
researchers wiU, in turn, depend upon 
educational professionals and families of 
students with disabilities who have been 
informed of the characteristics of valid 
research and are committed to asking for and 
using it in AT decision-making. Finally, 
information technologies will need to be 
developed and/or sustained to insure access 
to such information by educational and family 
consumers. 

This work was supported in part by U.S. 
Department of Education Grant 
#H324E050016 awarded to Illinois State 
University. The opinions expressed herein do 
not necessarily reflect the position or policy of 
the U.S. Department of Education and no 
official endorsement by the Department 
should be inferred. 
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